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Studies which have utilized lowerror-rate linear type programs have not been 
able to compare the effectiveness of various modes of iPeedback in correcting error 
in programed learning. In the present study using 75 university students, it was 
possible to correct errors without teaching erroneous material by using materials 
designed to teach 30 commonly misunderstood concepts in general science by means 
of computer-assisted instruction (CAI). The five treatment groups differed only with 
respect to feedback modes which were no feedback; feedback of correct or 
“wrong": feedback of the correct response; feedback appropriate to the student s 
response; and a combination of modes two. three, and four. Items missed were 
presented repeatedly until a criterion of correct response to each item had been 
attained. There are indications that the subjects who received feedback guiding them 
to the correct response were learning more effectively and performed better than 
those who were forced ^o "discover" the correct response. Data indicate that for 
correcting error, providing a student with a statement of which response was correct 
or why the correct response was correct may be the most valuable. The analysis of 
variance on posttest scores indicates that a combination of modes is slightly superior 
to some of the individual feedback modes in affecting immediate retention (MT) 
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under this cover the scholarly work of its professors. The search for truth 
and educational wisdom is truly one that involves all of us, and efforts such 
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One of the marks of a true professional is a wilhngness to share the 
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A COMPARISON OF SEVERAL FEEDBACK METHODS 



FOR CORRECTING ERRORS BY 
COMPUTER-ASSISTED INSTRUCTION 

David Alan Gllnian 
Indiana State University 

Computer-assisted Instruction (CAI) differs from programmed texts in 
that the student *s responses are evaluated against anticipated answers stored 
in the memory of the computer and feedbick appropriate to the student's 
response can be provided. Feedback and prompting the student to respond 
correctly can Identify and correct specific student errors. This may be a.i 
Important advantage for CAI over other types of Instruction. 

Prior studies in programmed learning have not been able to compare the 
effectiveness of the several modes of feedback in correcting errors because 
these studies utilized low error rate linear type programs. Since few 
Incorrect responses are made by « student learning by means of a typical 
linear program, little has been learned concerning how feedback can be used 
to correct errors. 

The correction of errors by providing the student with knowledge of 
results is also one of the goals of the adjunct auto-instruction techniques 
developed by Pressey. These techniques do not necessitate a low error rate 
program and thus provide a better format for investigating the use of feedback 
to correct learner errors. 

Another reason for the paucity of studies in error correction is an 
ethical consideration. Programmed learning researchers are very reluctant 
to teach learners Incorrect or even inaccurate Information so that they can 
systematically study how to correct errors* 
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This study used materials designed to teach commonly misunderstood 
concepts in general science. Errors made by the Ss occurred as a result of 
misconceptions they had acquired as a result of conventional instruction. It 
was thus possible to correct £s* errors without teaching erroneous material 
and without intentionally tricking them into committing errors. 

Rationale 

The use of knowledge of results as a mode of feedback has its basis in 
the principle that reinforcement of correct responses enhances learning. Many 
programmers using the knowledge of results technique believe that Its sale 
value is in its reinforcement qualities and that reinforcement occurs only 
when the student's response Is correct. 

Other programmers use feedback as a means of providing information to 
correct the student's misunderstanding. If there were no purpose to feedback 
other than to provide the student with reinforcement, statements such as 
are correct" should prove equally effective as a confirmation of the correct 
answer. 

However, there Is some evidence (Glaser, 1966) that providing the correct 
answer following an incorrect response is a reinforcing event In the same way 
as confirmation of a correct response. 

Bryan and Rlgney (1956) demonstrated the benefit of providing feedback 
contingent on the student's response. Two groups learned ship aperatlons by 
means of a tab test. When tested one week after training, an explanation of 
choice group was significantly superior to a knowledge of results group. 

It Is possible that there is some advantage in providing Ss with a 
combination of feedback modes in order to take advantage of reinforcement and 
at the same time provide the student with Information However, Svets and his 
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co-workers (1962) found that: "fairly extensive feedback may be detrimental 
to learning." Extensive feedback may also be inefficient in terms of time, 
since lengthy feedback messages require greater amounts of time. 

Procedure 

Seventy-five university upperclassmen were taught 30 general science 
concepts by means of a computer-assisted adjunct auto-instruction program. 

The frames of the program were multiple-choice items dealing with 30 general 
science concepts. One response to each item was a correct response* one 
response to each item was a conmon misunderstanding of the concept, and the 
other two responses were reasonable and plausible dis tractors. 

Equipment was an IBM 1410 computer and four IBM 1050 teletypewriter 
terminals equipped with random access slide projectors. Instruction was 
teleprocessed one-half mile between the terminals and the computer. The 
treatment groups differed only with respect to feedback modes. The five modes 
of feedback compared were (Group A) no feedback* (Group B) feedback of 
"correct" or "wrong," (Group C) feedback of the correct re:fponse choice, 

(Group D) feedback appropriate to the student's response, and (Group E) a 
combination of the feedback modes of Groups, B, G, and B. 

§8 were assigned to 15 strata on the basis of scholastic aptitude test 
scores. The five Ss in each strata were randomly assigned to one of the five 
treatment groups. The first iteration of the 30 item program served as the 
pretest and also provided instruction by means of feedback according to treat- 
ment group. The program caused all of the items to be presented on the first 
iteration, those missed on the first iteration to be presented on the 
second iteration, those on the second iteration to be repeated on the third 
iteration, etc., until a criterion of a correct response to each of the 
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thirty items had been attained* A paper and pencil posttest of 30 items 
similar to those of the program was administered following the instruction. 

A treatment x levels analysis of variance was performed to determine 
whether differences existed between any of the treatment groups with respect 
to the nund>er of responses required to attain criterion) the number of 
iterations of the program, time required to attain criterion, and posttest 
score* When significant differences were found, Tukey's W-Procedure was 
used to ascertain whether differences existed between specific pairs of means* 

Results 

Independent variables . Analysis of variance showed no difference 
(p > .05) for between means of treatment groups with respect to either the 
independent variable, scholastic aptitude or the number of correct responses 
on the first iteration of the program which served as a pretest as well as an 
Instructional program* 

Number of responses to criterion * The analysis of variance for the number 
of responses required for S,s to reach criterion (Table 1) shows an F-ratio 
for treatment effects clearly statistically significant (F “ 65*83) at the *01 
level* The results of the Tukey W«Procedure indicated that the means of 
Groups C, D, and E were each significantly better than those of Groups A and B. 

Number of iterations of program to criterion * The range of iterations of 
the program required by an £ to attain criterion were from two iterations for 
several S,s in Groups D and E to seven iterations for one S in Grcap A* 

The data from the number of iterations to criterion (Table 2) show an 
F-ratlo (F * 37*44) a clearly significant at the *01 level* 

The results from the Tukey W-Procedure found in Firt C of Table 2 again 
show statistically significant differences between each of the means of Groups 
C, D* and E and those of Groups A and B* 
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Time required t o complete instruction . Due to the relatively slow 
(abore 100 words per minute) typing rate of the 1050 terminal, those groups 
which received longer feedback messages (Groups D and E) naturally required 
longer to complete the instruction. 

The data from time required for Ss to complete the first iteration of 
the 30 item program found in Table 3 shows a high F**ratio for treatment 
effects (F « 32.70) which was statistically significant at the .01 level. 

Tukey*s W-Procedure shotted the anticipated result that those treatment 
groups which received long typed feedback messages (Groups D and E) required 
significantly longer to complete the thirty items than those groups which 
received short feedback messages (Groups B and C) and Group A which received 
no typed feedback messages. In each case, the significance was at the .01 

level. Differences between means of other groups were not significant at the 
.05 level. 

Tims ^ criterion The amount of time required for Ss to attain 
criterion, analyzed in Table 4, was significantly lower (p < .01) for Group C 
than for the other treatment groups and significantly higher (p< .01) for 
Group D than for any group except Group E. Differences between pairs of 
means of all other treatment groups were not significant at the .05 level. 

Correct responses on post test . The analysis of the number of correct 
responses on the posttest is found in Table 5. Analysis of variance results 
show an P-ratio for treatment effects (F “ 3.97) statistically significant 
at the *01 level. The Tukey W“Procedure showed significant differences 
between the means scores of Group E and those of Groups A, B, and C. Although 
the mean of Group D was higher than the means of Groups A, B, and C, there 
was no significant difference between any two of these means. 
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Level effects . The F-ratio for level effects was not significant at 
the .05 level for any of the four dependent variables. 

Discussion. Conclusions and Recommendations 

Rate of learning . In terms of the results obtained in the analysis of 
number of responses .and iterations of the program required by £s to reach 
criterion, there strong indications that Ss who received feedback guiding 
them to the correct response were learning more effectively and performed 
better than did those who were forced to "discover” the correct 
response. The results and their level of significance clearly Indicate 
the value of providing information to students during a programmed instruction 
feedback. The findings are in agreement with those of Hollrnd (1966) who 
concluded that there were no advantages for prompting a student to give the 
correct answer after an error had been committed. Holland concludes that if 
a student does not know the correct answer, he might as well be told it. 

These findings differ with the point of view of those programmers who 
prefer the simple knowledge of results technique and who find no advantage in 
showing the correct answer to learners who provide incorrect responses. This 
study indicates that simple st?^.-ffients such as "you are correct" do not prove 
equally effective as revealing the correct answer. Also, this study indicates 
that the appearance of a correct answer is not wasted when the student's 
response was incorrect. Data from the present study, however, indicate that 
providing a student with a statement of which response was correct, or 
providing him with a statement of why the correct response is correct may be 
of much more value than merely telling him "correct" or •‘wrong." The poor 
results demonstrated by the knowledge of results feedback group (Group B) 
raise questions as to whether this mode of feedback is of much value for the 
correction of errors. 
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In the comparisons of feedback mode and learning rate, it is Interesting 
to note that there were no significant differences between Groups C, D, and 
B« Apparently the factor which accelerated the learning of S_s was being 
informed as to which response was the correct one* In both comparisons, 
however, the ffjean of Group B, the combination of fec>dback modes group, was 
only slightly, but not significantly better than the means of Groups C and D, 
and in both cases, significantly better than the means of Groups A and B* 

This finding is contrary to those of Swets and his co-workers (1962) who 
found that fairly extensive feedback may be detrimental to learning* 

Time required for instruction * The time required for a student to 
receive instruction by CAT is a function of the number of instructional 
frames he completes and also is a function of the amount of time the student 
terminal spends transmitting messages* 

Several studies (Gilman, 1967b; Wodtke and Gilman, 1966) have demon- 
strated that the operating speed of a teletypewriter terminal is slower than 
would be ideal for an interface between student and computer* 

lamedtate retention * The analysis of variance conducted on posttest 
scores indicated that the combination of feedback modes (Gro;ip E) was superior 
to some of the feedbac' tups in terms of number of correct responses* 
Apparently the amount c£ information the ^ derives from the feedback is 
important in affecting retention* The results of the present study indicate 
the advantages for learning attained by providing the correct response when a 
learner makes an error and also show the advantages for retention in 
providing the S with extensive information in feedback messages* 

Recoainendations for Further Research 

Further research is necessary to determine the effects of using the 
various modes of feedback to correct errors* F*. ly forms of programmed 
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learning required the learner to reveal, by making some form of error^ the 
kind of instruction he should receive next* However, most studies have been 
conducted with relatively error free programs and little is presently known 
concerning how to correct errors in instruction* 

The present study should be repeated using an interface capable of 
faster communication and response time than the 1050 terminal. Also, the 
present study should be repeated using a delayed retention measure in 
addition to the Immediate retention measure* 

At Indiana State, we are continuing efforts in computer -as sis ted 
instruction research and hope to develop computer-controlled multi-media 
learning centers to be used in future computer-assisted instruction. 
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TABLE 1 

ANALYSIS OF DATA FOR SOURCES OF VARIATION FROM 
NUMBER OF RESPONSES TO CRITERION 





A. Group Means 






Group A 
Means (n = 15) 


Group B Group C 

(n « 15) (n « 15) 


Group D 
(n « 15) 


Group E 
(n ® 15) 


Grand Mean 
(» - 75) 


74.53 


71.93 54.20 


54.00 


50.47 


61.03 



B. Analysis of Variance 



Source of 




Sums of 


Mean 


F 




Variation 




Squares 


Square 


Ratio 


Significance 


Treatments 


4 


7,633.15 


1,908.29 


65.83 


(p .01) 


Levels 


14 


689.55 


49.25 


1.90 


n.s. 


Treatments X 












Levels 


56 


1,623.25 


28.99 






Total 


74 


9.945.95 









C. Tukey*s W«Procedure for Differences 
Between Pairs of Means 



Group A 
Group B 
Group C 

Groap D 



Group B 
2.60 



Group C 

20.33** 

17.73** 



Group D 

20.53** 

17.93** 

0.20 



Group E 

24.06** 

21.46** 

3.73 



.05%, 56 = 5.50 

.01%, 56 = 6.67 

**Slgnificant 
(P - .01) 
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TABLE 2 

ANALYSIS OF DATA FOR SOURCES OP VARIATION FROM 
NUMBER OP ITERATIONS OP PROGRAM TO CRITERION 





A. Group Means 






Group A 
Means (n “ 15) 


Group B Group C Group D 

(n ® 15) (n ■ 15) (n ® 15J. 


Group E 
(n - 15) 


Grand Mean 
(H - 75) 


4.67 


4.60 2.73 2.87 


2.53 


3.48 



B. Analysis of Variance 



Source of 
Variation 




Sums of 
Squares 


Treatments 


4 


67.38 


Levels 


14 


5.92 


Treatemnts X 
Levels 


56 


25.41 


Total 


74 


98.72 



Mean 


P 




Square 


Ratio 


Sisnificance 


16.85 


37.44 


(p ^ .01) 


0.42 


1.00 


n.s. 


0.45 







C. Tukey’s W-Procedure for Differences 
Between Pairs of Means 





Group B 


Group C 


Group D 


Group E 




Group A 
Group B 
Group C 
Group D 


0.07 


1.94** 

1.87** 


1.80** 

1.73*v^ 

0.14 


2.14** 

2.07** 

0.20 

0.34 


.05”S,56 « 0.69 
.01%. 56 • 0.84 
**Signif leant 

(p^ .01) 
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TABLE 3 

ANALYSIS OF DATA FOR SOURCES OF VARIATION FROM 
CORRECT RESPONSES ON POST-TEST 



A. Group Means 



Group A 


Group B 


Group G 


Group D 


Group E 


Grand Mean 


Means (n =* 15) 


(n = 15) 


(n - 15) 


Cn =.151 


(n « 15) 


(N “ 75) 


25.87 


25.73 


25.80 


27.60 


28.67 


26.73 



B. Analysis of Variance 



Source of 




Sums of 


Mean 


? 




Variation 


a 


Squares 


Square 


Ratio 


Significance 


Treatments 


4 


106.67 


26.67 


3.97 


(P C .01) 


Levels 

Treatments X 


14 


80.27 


5.73 


1.00 


n.s. 


Levels 


56 


375.73 


6.71 






Total 


74 


562.67 









C. Tulcey*s W-Procedure for Differences 
Between Pairs of Means 



Group A 
Group B 
Group C 
Group D 



Group B 
0.14 



Group C 

0.07 

0.07 



Group D 

1.73 

1.87 

1.80 



Group E 

2.80* 

2.95* 

2.87* 

1.07 



% 



•05”5,56 » 2.67 

.01^5.56 «= 3.21 

*Significant 
(p ,05) 
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TABLE 4 

ANALYSIS OF DATA FOR SOURCES OF VARIATION FROM 
TIME FOR FIRST ITERATION OF PROGRAM 



A* Group Mfeans 



Mean 

(in 


Group A 


Group B 


Group C 


Group D 


Group E 


Grand Mean 


min* 


It- 151 


(n - 15) 


(n ■ 15) 


(n = 15) 


(n « 15) 


(N - 75) 


utes) 


26.36 


27.80 


29.58 


47.10 


44.44 


35.00 



Analysis of Variance 



Source of 




Sums of 


Mean 


F 




Variation 


df 


Squares 


Square 


Ratio 


Significance 


Treatments 


4 


5,862.40 


1,465.60 


32.70 


(p<- .01) 


Levels 


14 


594.30 


42.75 


1.00 


n.s. 


Treatments X 








' 




Levels 


56 


2,509.92 


44.82 






Total 


74 


8,966.62 









C# Tukey*s N-Procedure for Differences 
Between Pairs of Means 





Group B 


Group C 


Group D 


Group E 




Group A 


1.44 


3.22 


20.74** 


18.08** 


.05*'5,56 - 6.91 


Group B 




1.78 


19.30** 


16.64** 


Group C 






17.52** 


14.86** 


.0lW5,56 » 8.38 


Group D 








?a66 



^Significance 
(P ^ .01) 




TABLE 5 



ANALYSIS OF DATA FOR SOURCES OF VARIATION FROM 

TIME TO CRITERION 



A« Group Means 



Mean 

(in Group A Group B 

min- (n « 15) (n 15) 

utes) 

57.21 56.12 



Group C 
(n « 15) 



Group D 
(n ”15) 



Group E 
(n « 15) 



Grand Mean 
(N » 75) 



44.70 



69.00 



64.02 



58.21 



B. Analysis of Variance 



Source of 




Sums of 


Mean 


F 




Variation 




Squares 


Square 


Ratio 


Significance 


Treatments 


4 


5,070.00 


1,267.50 


19.03 


(p < .01) 


Levels 


14 


1,842.40 


131.60 


1.95 


n.s. 


Treatments X 












Levels 


56 


3,730.72 


66.62 






Total 


74 


10,643.12 









C. Tukey*s W-Procedure for Differences 
Between Pairs of Means 





Group B 


Group C 


Group D 


Group E 


Group A 


1.09 


12.51** 


11.79** 


6«81 


Group B 




11.42** 


12.88** 


7.90 


Group c 






24.30** 


19.32** 


Group D 








5.81 



.05”s,56 8.41 

,01%, 56 - 10.20 



**Significant 
(P < .01) 
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TABLE 6 

COMPUTER-STUDENT INTERACTION 
SAMPLE PROGRAM ITEMS 



QUESTION (on slide) 

01* Which of the following statements describes the relationship between 
insects and insecticides? 

a) species of insects develop imnunlty to insecticides through 
natural selection 

b) individual insects can build up a resistance to an insecticide 
through repeated contact 

c) insecticides no longer have any effect on Insects 

d) insecticides have little effect on insects 



COMPUTER-STUDENT INTERACTION (on teletypewriter) 

Group A: No Feedback 

Correct Response Incorrect Response 



Computer : 
Student: 


01. 


a 


Computer: 

Student: 


01 


Computer: 


02. 




Computer: 


02 



Group B: Knowledge of Results 

Correct Response 

Computer: 01* 

Student; a 

CompuTvsr; Correct 

02 . 



Group C; Knowledge of Correct 

Correct Response 

Computer: 01. 

Student: a 

Computer: a is correct 

02 . 



Feedback 



Incorrect Response 







Computer: 

Student: 


01. 

b 


Computer: 


Wrong 

02. 


Response Feedback 




Incorrect Response 


Computer: 

Student: 


01. 


b 


Computer: 


a is correct 
02. 
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Group D: Response Contingent Feedback 



Correct Response 



Computer: 

Student: 

Computer: 



01 . 



a 



Certain species o£ 
Insects are sometimes 
capable of developing) 
through natural selec- 
tion) an Immunity to 
insecticides. 

02 . 



Incorrect Response 



Computer; 

Student: 

Computer: 



01 . 



The individual Insects 
themselves are not able to 
build up resistance to 
insecticides. Certain 
species of insects are 
sometimes capable of de- 
veloping) through natural 
selection, an Itonunity to 
insecticides. 

02 . 



Group E: combination of Feedback Modes 

Correct Response 



Computer: 

Student: 

Computer: 



01 . 



a is 



Correct, 
correct. 

Certain species of 
insects are sometimes 
capable of developing, 
through natural selec- 
tion, an immunity to 
insecticides. • 

02 . 



Incorrect Response 



Computer : 

Student: 

Computer: 



01 . 



Wrong. The individual 
Insects are not able to 
build up resistance to 
insecticides, 
a is correct. 

Certain species of insects 
are sometimes capable of 
developing, through 
natural selection, an 
immunity to insecticides. 
02 . 



